Randomly incomplete spectra and intermediate statistics 
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By randomly removing a fraction of levels from a given spectrum a model is 
constructed that describes a crossover from this spectrum to a Poisson spectrum. The 
formalism is applied to the transitions towards Poisson from random matrix theory 
(RMT) spectra and picket fence spectra. It is shown that the Fredholm determinant 
formalism of RMT extends naturally to describe incomplete RMT spectra. 

It is by now well established that, in the semiclassical limit, spectral properties of 
physical systems whose underlying classical motion is chaotic are consistent with the 
predictions of the Wigner-Dyson random matrix theory while those with an underlying 
regular motion behave as an uncorrelated sequence of levels (Poisson statistics) (see Q 
for a review). Matrix models have been proposed to describe transitions between these 
two extreme behaviors. In these interpolating matrix models the transition is generated 
by reducing the magnitude of the off-diagonal elements with respect to the diagonal 
ones. This can be achieved in different ways, for instance by decreasing the variance of 
all off-diagonal elements^, by making the matrices bandedji'l or by introducing a power- 
law decay 6] or more complicated schemesfT^. In this kind of models the invariance of 
the ensemble probability distribution under unitary transformation is broken. Spectral 
distributions of some invariant ensembles may also show deviation from the Wigner-Dyson 
statistics but without reaching the Poisson limit Apparently symmetry breaking is 
necessary to completely eliminate the correlations among levels[l(|. 

Despite that classical analogy does not support the assumption of a unique crossover 
from chaotic to regular regimes, the existence of a third universal statistics has been con- 
jectured motivated by the Anderson model description of the metal-insulator transition 
(MIT)^3l- In the MIT, localized states of the insulator phase obey Poisson statistics 
while the extended states of the metal phase show a Wigner-Dyson behavior. The third 
universal behavior would correspond to statistics observed at the critical point. The main 
characteristics of this behavior would be the multifractality of the eigenstates, nearest 
neighbor spacing distribution (NND) exhibiting a linear level repulsion with a slope at the 
origin steeper than the Wigner case and with an exponential decay for large separations 
in contrast to the Gaussian standard decay. Long range statistics like number variance 
would increase linearly as in the Poisson case but with a smaller slope Although 
ensembles have been found, invariant or not, that show some of these properties, the 
extent of their validity has not yet been established. 

Perhaps the simplest model with these spectral features is the short range plasma 
model jl3|. It consists in a ID Coulomb gas model for which the range of the interaction 
is restricted to a finite number of neighboring levels. An appealing feature of the model 
is its amenability to analytical treatment. It was found that the intermediate statistics 
obtained when only adjacent pairs interact, denoted by the authors as the semi- Poisson 



statistics, gives an approximately good description of spectral properties of some diffrac- 
tive billiards [14]. 

It has been shown that semi-Poisson statistics can also be obtained in a completely 
independent way as a particular case of a family of statistics termed by their authors 
the daisy models It consists in removing every other r levels from an uncorrelated 
spectrum. The particular case r = 1 (every other level) corresponds exactly to the semi- 
Poisson statistics. Notice also that it has been known for quite a long time that spectral 
properties of two different classes of spectra can be related by the operation of dropping 
levels from one of them, namely measures of the symplectic and the orthogonal ensembles 
of RMT0 can be connected in this way. 

In the above cases, levels are removed in a correlated way (every other level) and 
the operation results in a more correlated spectrum (from GOE to GSE, from Poisson 
to semi-Poisson). Our purpose here is to investigate the same operation but performed 
randomly. Specifically, we consider an infinite spectrum and, after dropping at random 
a fraction 1 — / of levels, the remaining fraction / is studied (0 < / < 1). In order 
to keep to unity the average level density the remaining spectrum is correspondingly 
contracted. To determine the statistical properties of the transformed spectra we resort 



to the formalism developed in our previous work|17l| in which the problem of randomly 
incomplete spectra was considered. We showed that when / — > 0, irrespective of the 
nature of the initial spectrum, the statistical properties approach those of an uncorrelated 
spectrum (the Poissonian spectrum is a fixed point of the operation). Therefore, the 
random dropping operation generates models whose statistics are intermediate between 
those of the initial spectrum and Poisson statistics. Obviously it can also be seen that the 
statistical properties of a Poisson sequence are not affected by this dropping operation. 

In the present communication we discuss properties of models constructed starting 
with RMT spectra and with a picket fence of equally spaced levels. We show that in 
the first case a family of intermediate statistics parameterized by the fraction / of the 
remaining levels is generated that shows features similar to those of the intermediate 
critical statistics. In particular, the / = 1/2 case is compared with the semi-Poisson 
statistics. In the picket fence model we show that the / = 1/2 member of the family 
reproduces the spectral properties of a sequence of levels weakly confined by a log-normal 
potential (see below). 

For later use, we recall some notations and results of [l^. We consider a spectrum 
p{E) = ^ 6{E — Ei) with mean spacing < p >= 1. The two-point cluster function 
Y2{x) = 1— < p{E — x/2)p{E + x/2) > gives the disconnected part of the two-point 
correlation function. One has the basic relation 

m (x) = Y, (^j^ (1) 

that expresses the two-point cluster function of a spectrum with a fraction 1 — / of 
missing levels in terms of the same function of the complete spectrum. We use capital 
letters to denote the quantities of the complete intial spectrum and small cases with 
a superscript for the incomplete ones. Similar scaling relations hold for higher-cluster 
functions. From (Q) other statistical measures can be easily derived. For instance the 
form factor K (t) = 1 — B{t), where B{t) is the Fourier transform of the cluster function. 



transforms as 



fc(r) = l-/ + /ir(/r). (2) 

Similarly, the number variance (variance of the number of levels contained in an 
interval of length L) of the transformed spectrum is expressed in terms of the same 
quantity of the complete spectrum 

aHL) = {l-f)L + fE'(^]. (3) 



The important feature of this relation is the appearance of a linear term suggesting the 
same behavior as for critical statistics. In particular, the Poisson expression is recovered 
when / — > 0. 

Another set of statistical measures are the E{n, s) functions (n-level probability func- 
tions) that give the probabilities of finding n levels (n = 0, 1,2...,) inside a segment of 
length s. If their expressions E{n, s) for the complete spectrum are known then the first 
{n = 0) of these functions (gap probability function), when only a partial fraction / of 
levels taken at random remains, is given by 

oo 

e{sJ) = J2i^-f)'m^), (4) 

k=0 •' 

which follows from the fact that 1 — / is the probability that one level was dropped. By 
the same argument the NND p{s, f) is given by 

oo 

p{s,f) = Y.{l-ffP{k,^), (5) 

k=0 •' 

where the P{k, s) are the density distributions of the spacings between two levels contain- 
ing k levels inside the complete sequence (P(0, s) is the NND of the complete sequence, 
p{s,f) the one corresponding to the incomplete sequence). This expression was first 
proposed as an annate [l^ and in Ref. [l^ it is shown that the coefficients / (1 — f)'^ 
maximizes Shannon entropy with constraints appropriately defined. 

The above equations show that these expressions for the gap probability and NND 
of the transformed spectrum are the generating functions of all the n-level probability 
functions and spacings distributions of the complete spectrum. Indeed, by defining the 
generating function 



G{t,z) = J2i-^)'i^-^fE{k,t) (6) 

fc=0 

such that 

-1)" 



E{n,t) 



dG{t,z) 



(7) 



then, from Eq. (jlj), the identification e(s, /) = G(j, /) can be made. Of course, an 
equivalent identification also holds for the incomplete NND given by Eq. (0). 



As a trivial example of the above relation, the expressions E{n,s) = ^exp(— s) for 
the Poissonian n-probability functions are generated by the function G{t, z) = exp{—tz). 
Taking t = j and z = f we can check that they are not affected by the dropping level 
operation. 

We apply now the above formalism to initial spectra of standard RMT. By observing 
that the e-function is the generating function of the n-level probability functions we can 
establish connections between the statistical properties of the transformed spectra and 
the RMT Fredholm determinants. These are the determinants D{t,z) = det (1 — zK) 
and D±{t,z) = det (1 — zK±) where K and K± are, respectively, the integral operators 
with kernels K(x, y) = ^ sm{x — y)/{x — y) and K± {x,y) = K{x, y) ± —y) defined 
on L2([0,7rs])|i3. 

Starting with the unitary Gaussian ensemble (GUE), the n-probability functions (in 
the sequel, the index [3 with /5 = 1, 2 and 4 denotes quantities of the orthogonal, unitary 
and symplectic ensembles, respectively) are given in terms of D{t, z) by 



E2{n,t) 



dD{t,z) 



J 2 = 1 



For GOE the even and the odd n-probability functions are given by 



n-l 



El {2n, t) = Y,E^{k,t)-Y,E_{k,t) 



(9) 



fc=0 



fc=0 



with £;_(-!, t) = and 



Ei(2n + l,t) = ^[E^{k,t) - E+{k,t)] 



(10) 



fc=0 



where E± are expressed in terms of D± as E2 and D in (jHI). In terms of E±, the symplectic 
n-probability functions are given by 



Ei{2n,t) = - [E+{k,2t) + E_{k, 2t)] 



On the other hand Jimbo et al. [203 have shown that the determinant D{t, z) is given by 



\iiD{t,z) = / ^ ' ^ dx, 
Jo X 



(12) 



where a{x, z) is the solution of the differential equation 



X 



fa 

dx"^ 



+ 4 ( - a 

dx 



X 



da 

dx 



fda\ 
\dx J 



(13) 



that satisfies the boundary condition a{x, z) ^ 
that D±{t, z) are given in terms of D{t, z) as 



-X when x ^ 0. It has also been shown 



1 I r 

hiD±{t,z) = -\YiD{t,z)±- dx^--\nDit, 



(14) 



From the above we derive that the incomplete gap functions in the three cases are 
connected with the respective determinants D{t, z) and D±{t, z) by the relations 



for the unitary, 



e^{sJ)=D{-J) 



D^i^,2f-n + {l-f)D,i^,2f-n 



for the orthogonal, after some algebra, and finally for the symplectic 



e4(y,/) = - 



(15) 



(16) 



(17) 



The above equations provide exact expressions for the gap functions of both complete 
and randomly incomplete RMT spectra. They are one of the main results of the present 
communication as they provide a physical interpretation for all the real values of the 
parameter z in the interval [0, 1] appearing in the Fredholm determinant. With z = 1 
they have been used to derive asymptotics of the spacings for large separations |21i]. We 
now show that when spectra are incomplete a major change in their asymptotics appears. 
This follows from the fact that with < z < 1, the function a(x, z) behaves, when x oo, 
asET 



X 



aix, z] 



TT 



ln(l - z) 



Substituting into the above equations, one finds that to leading order 



e/3(s,/) = exp 



5ln(l 



/) 



1. When / 



(18) 
(19) 

one gets the 



for P = 1,2 while / has to be multiplied by 2 for P 
Poisson behavior exp(— s) irrespective of the value of f3. 

Let us remark that although we discuss here the e-function, the identification of the 
incomplete NND p-function as a generating function of spacing distributions can also be 
used to perform a similar analysis ji^. 

We can now compare the incomplete GOE case with the semi-Poisson model. This 
model gives rise to a linear number variance with slope 1/2. By taking / = 1/2 in Q, 
the incomplete GOE spectrum has the same behavior for large values of L apart from a 
small contribution of the term. Considering the form factor we have for the incomplete 
sequence 



kir) = 1 - / + / 
and for the semi-Poisson model 



2/r + /rln(l + 2/r), < /r < 1 
2-/rlnfi|^V /r>l 



2 + ttV^ 



(20) 



(21) 



4 + 7r2r2' 

In Fig la these two functions are compared. They do not coincide but they show strong 
similarities: they both start with a value 1/2 at the origin, reflecting an identical (lack of) 



rigidity and for large r they both tend to 1 hke Concerning the spacing distributions, 
recall first that the semi-Poisson NND is given by 4gexpf— 2s) jl3l|. Close to the origin 
only the k = term contributes in therefore p{s, f) ~ -Pgoe(0, y) ~ leading 
with / = 1/2 to a slightly smaller slope for the incomplete spectrum. On the other hand, 
taking / = 1/2 in (fTU|) we have for large separations a decay exp [— 2(ln2)s] slower than 
the semi-Poisson one. This comparison is illustrated in Fig lb. In summary, although 
with similarities, the incomplete GOE model presents differences with respect to the 
semi-Poisson model. 

It is worth noticing that although our procedure interpolates between initial and final 
(Poisson) spectra, it gives rise to different results from the ones resulting from superpos- 
ing in an uncorrelated way different spectra^^. In this latter case, for example, level 
repulsion is destroyed while the dropping mechanism presented here preserves it for all 
values of /. 

Let us now apply the dropping procedure to a picket fence spectrum defined as a 
sequence of points located, say, at ... — 3/2,-1/2,1/2,3/2.... From the definition, one 
can write 

P{n,s)=5{s-n-l) (22) 

and 

^^^'^) = | 0, \s-n\>l ^23) 

and for the two-point cluster function 

oo 

^2(0;) = l-5^5[a;-(n + l)] (24) 

n=0 

which follows from and the general relation Y2{x) = 1 — Yl,'^=oPi.^y^)- The picket 
fence is the most 'correlated' spectrum and its rigidity reflects, for instance, in the small- 
ness of the number variance 

{L) = L- [L] - (L - [L])' (25) 

where [L] stands for the integer part of L. 

For later comparison with the behavior of weakly confined eigenvalues, consider now 
what we denote as the 'continuous' version of the picket fence spectrum, namely each 
point is randomly, independently and uniformly distributed inside an interval of unit 
length around the values ... — 3/2, —1/2, 1/2, 3/2, .... In this case, the 5-functions in 
of the spacing distributions become the "triangles" 



0, s < n 

, . I s-n, n<s<n + l , . 

^^"'^) = ^ s-n-2, n + l<s<n + 2 (^6) 
0, n + 2 < s 



that preserve the normahzation < 1 >= 1 and < s >= n + 1. From (j26|) we find that 
the cluster function takes the simple expression 



from which follows 



Yo(x) 



0<x<l 
0, Kx 



(27) 



K{t) 



sm vrr 



TCT 



and 



L-L'^ + < L < 1 

1/3, 1 < L. 



:28) 



(29) 



Using the general relations for incomplete spectra we find for the transformed cluster 
function 



y2[x) 



1 - x/f, 0<x<f 
0, f<x, 



form factor 



Hr) = 1 - / 



sin(/7rr) 
fnr 



1 2 



and number variance 



a 



/3, / 



Finally, substituting (j26p into ©, the NND for the incomplete sequence becomes 



p{s,f) 



f-'s, 0<s<f 
l-fr-'il + nf-s), nf<s<in + l)f, n = 1,2,3. 



(30) 



(31) 



(32) 



(33) 



(see Fig 2, for illustration). 

Let us compare these results with the behavior of the eigenvalues of matrix ensembles 
in which the eigenvalues are confined by a weak log-normal potential. By that is meant 
eigenvalues of NxN random matrices M distributed according to 



P(M) = exp [-TrV{M)] (34) 

with V{x) = In^ \x\ /•y. It has been shown in Ref. that in the limit of weak confinement 
7 — s> oo, the spectrum, whose levels tend to locate around the sites of a crystal 
lattice, has, after unfolding, the following structure. Construct 2N intervals of length 1/2. 
symmetrically with respect to the origin. The N levels occupy randomly these intervals as 
follows: i) in an interval there is at most one level, ii) intervals symmetric with respect to 
the origin can not be simultaneously occupied. This last property introduces long range 
correlations and lack of stationarity (translation invariance). By performing an average 
over the spectrum or, equivalently, by considering only the first (or second) half of the 



spectrum, the effect of this long range correlations is washed out and one obtains, for the 
two-point cluster function 



ViW-r^'S'.JI^f^' (35) 



and for the NND (see j9|) 



P(s) - I 2., < . < 1/2 

\2-"+i(l+r2/2-s), n/2<s<(n+l)/2, n = l,2,3.... ^ ^ 

Obviously, (jHKjl and are just (pUjl and with / = 1/2, corresponding to the 
continuous picket fence for which half of the levels have been randomly removed (see Fig. 
2). 

Let us finally mention an example of a different sort of mechanism than dropping or 
removing points. Here points are located on a line in the plane and, as a parameter is 
varied, a fraction of them leave the line. Specifically, consider the roots of a random 
polynomial of degree N 

N 

Pn{z) = Y,(^uz^ (37) 

k=0 

where the a^'s are independent Gaussian complex random variables (real and imaginary 
parts centered at zero and variance (T^)j2^. If one imposes the symmetry (self-inversive 
(SI) symmetry, also called selfreciprocal or conjugate reciprocal) 

iN-k = Ofc, (38) 

where the bar denotes complex conjugate, one can see that the roots of Pn{z) lie either 
on the unit circle C or appear in pairs symmetrically located under inversion with respect 
to it. The relevant parameter of the model is e = a^/N. As e increases, some roots leave 
C and in the limit e oo, on the average, a fraction = l/-\/3 of the roots remains on 
it 0,11^. On the other hand, in Ref. [2y], the restricted class of SI polynomials having 
all the roots on C has been considered. It has been found that their statistical properties 
coincide with those of eigenvalues of the orthogonal ensemble (OE) of random matrices 
{(3 = 1). One may then ask wether properties of the unrestricted class of SI polynomials, 
in the limit e — oo, share some properties with the ones corresponding to dropping at 
random a fraction 1 — of zeros of the restricted polynomials. Consider, for instance, the 
NND. In both cases it starts linearly at the orig in, with a slope 7r^/6 for the restricted 
case and 7r^/(10\/3) for the unrestricted casej24|, which is five times smaller than what 
would result from randomly dropping a fraction 1 — of points from an OE sequence, 
namely 7r^(6</))^^ (see Eq. (0)). Leaving C has not the same effect as dropping randomly 
points on it. The points who move and locate on the complex plane "interact" with those 
remaining on C. 

In conclusion, by dropping at random a fraction of levels of a given spectrum a family 
of spectra is generated. Its statistical properties are intermediate between those of the 
initial one and a Poisson spectrum. Applied to eigenvalues of RMT, the family contains as 
a particular case a model exhibiting some of the features of critical intermediate statistics. 
Fredholm determinants of argument z are one of the basic structures appearing in RMT 
(one is usuUy interested in properties corresponding to z = 1). We show, for z < 1, 



that properties of Fredholm determinants characterize also the behavior of incomplete 
spectra and that z corresponds to the remaining fraction of levels. When the dropping 
procedure is applied to a picket fence spectrum we show that the generated family has the 
statistical properties of an ensemble of eigenvalues weakly confined. Finally we compare 
with a system for which, when a parameter is varied, a fraction of points on a line are 
not dropped but move on the complex plane. 

One of us (OB) thanks P. Forrester for an illuminating discussion. This work is 
supported in part by the Brazilian agencies CNPq and FAPESP. 
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Figure Captions 

Fig. 1 (a) form factor; (b) nearest neighbour spacing distribution (NND). Full line: 



incomplete (/ = 1/2) orthogonal ensemble; dashed hne: the semi-Poisson model; dotted 
line: orthogonal ensemble. 

Fig. 2 (a) form factor; (b) nearest neighbour spacing distribution (NND). Full line: 
incomplete (/ = 1/2) continuous picket fence; dotted line: orthogonal ensemble. 



